Skip to content

Conversation

@v1v
Copy link
Member

@v1v v1v commented Nov 30, 2020

What

  • Enable pipeline with matrix compatibility for the given ruby and frameworks. There is no exclusion yet.
  • Support JUnit reporting (similar to the approach done in the apm-agent-ruby with the yarjuf dependency.
  • Return error level for the docker-compose run
  • Use COPY directive to enable the caching for the docker build, since docker run overrides the /vendor folder.
  • Add CI Badge

Why

Use CI Jenkins to run the builds/tests for this project

Questions

@v1v v1v self-assigned this Nov 30, 2020
@v1v v1v added the automation label Nov 30, 2020
@ghost
Copy link

ghost commented Nov 30, 2020

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #4 updated

  • Start Time: 2020-12-02T09:30:50.201+0000

  • Duration: 11 min 59 sec

Test stats 🧪

Test Results
Failed 8
Passed 132
Skipped 0
Total 140

Test errors 8

Expand to view the tests failures

Tests / Matrix - RUBY_VERSION = 'ruby:2.3', FRAMEWORK = 'rails-6.0' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block (3 levels) in <module:EcsLogging>'"]
           
    

Tests / Matrix - RUBY_VERSION = 'ruby:2.3', FRAMEWORK = 'rails-5.2' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block (3 levels) in <module:EcsLogging>'"]
           
    

Tests / Matrix - RUBY_VERSION = 'ruby:2.3', FRAMEWORK = 'sinatra-2.0' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block (3 levels) in <module:EcsLogging>'"]
           
    

Tests / Matrix - RUBY_VERSION = 'ruby:2.3', FRAMEWORK = 'rails-5.1' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block (3 levels) in <module:EcsLogging>'"]
           
    

Tests / Matrix - RUBY_VERSION = 'jruby:9.1', FRAMEWORK = 'rails-5.1' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block in EcsLogging'"]
           
    

Tests / Matrix - RUBY_VERSION = 'jruby:9.1', FRAMEWORK = 'rails-6.0' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block in EcsLogging'"]
           
    

Tests / Matrix - RUBY_VERSION = 'jruby:9.1', FRAMEWORK = 'rails-5.2' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block in EcsLogging'"]
           
    

Tests / Matrix - RUBY_VERSION = 'jruby:9.1', FRAMEWORK = 'sinatra-2.0' / Tests / EcsLogging::Logger with a progname includes it – with a progname

    Expand to view the error details

     failed EcsLogging::Logger with a progname includes it 
    

    Expand to view the stacktrace

     
            
    expected: "yes"
         got: nil
    
    (compared using ==)
    
    ["./spec/ecs_logging/logger_spec.rb:76:in `block in EcsLogging'"]
           
    

Steps errors 8

Expand to view the steps failures

Shell Script

  • Took 1 min 34 sec . View more details on here
  • Description: [2020-12-02T09:39:08.094Z] + bin/dev -iruby:2.3 -frails-6.0 [2020-12-02T09:39:08.094Z] IMAGE_NAME=a

Shell Script

  • Took 2 min 40 sec . View more details on here
  • Description: [2020-12-02T09:39:48.227Z] + bin/dev -ijruby:9.1 -frails-6.0 [2020-12-02T09:39:48.227Z] IMAGE_NAME=

Shell Script

  • Took 1 min 34 sec . View more details on here
  • Description: [2020-12-02T09:39:22.505Z] + bin/dev -iruby:2.3 -frails-5.2 [2020-12-02T09:39:22.505Z] IMAGE_NAME=a

Shell Script

  • Took 2 min 37 sec . View more details on here
  • Description: [2020-12-02T09:39:53.891Z] + bin/dev -ijruby:9.1 -frails-5.2 [2020-12-02T09:39:53.891Z] IMAGE_NAME=

Shell Script

  • Took 1 min 40 sec . View more details on here
  • Description: [2020-12-02T09:39:54.212Z] + bin/dev -iruby:2.3 -frails-5.1 [2020-12-02T09:39:54.213Z] IMAGE_NAME=a

Shell Script

  • Took 1 min 29 sec . View more details on here
  • Description: [2020-12-02T09:39:38.276Z] + bin/dev -ijruby:9.1 -frails-5.1 [2020-12-02T09:39:38.276Z] IMAGE_NAME=

Shell Script

  • Took 1 min 39 sec . View more details on here
  • Description: [2020-12-02T09:39:53.072Z] + bin/dev -iruby:2.3 -fsinatra-2.0 [2020-12-02T09:39:53.072Z] IMAGE_NAME

Shell Script

  • Took 2 min 40 sec . View more details on here
  • Description: [2020-12-02T09:39:58.944Z] + bin/dev -ijruby:9.1 -fsinatra-2.0 [2020-12-02T09:39:58.944Z] IMAGE_NAM

Log output

Expand to view the last 100 lines of log output

[2020-12-02T09:41:41.404Z] 2 gems installed
[2020-12-02T09:41:41.404Z] Removing intermediate container 2d6695936ba3
[2020-12-02T09:41:41.404Z]  ---> c1c8239faf3c
[2020-12-02T09:41:41.404Z] Step 20/20 : WORKDIR /app
[2020-12-02T09:41:41.404Z]  ---> Running in 504198e69678
[2020-12-02T09:41:41.404Z] Removing intermediate container 504198e69678
[2020-12-02T09:41:41.404Z]  ---> d1600d770390
[2020-12-02T09:41:41.404Z] 
[2020-12-02T09:41:41.404Z] Successfully built d1600d770390
[2020-12-02T09:41:41.404Z] Successfully tagged apm-agent-ruby:ruby_2.4
[2020-12-02T09:41:41.404Z] IMAGE_NAME=apm-agent-ruby:ruby_2.4 USER_ID_GROUP=1154:1155 docker-compose run --rm specs 
[2020-12-02T09:41:41.404Z] Creating network "ecs-logging-ruby_default" with the default driver
[2020-12-02T09:41:41.404Z] Creating volume "ecs-logging-ruby_vendor" with default driver
[2020-12-02T09:41:41.689Z] Post stage
[2020-12-02T09:41:41.705Z] Recording test results
[2020-12-02T09:41:42.361Z] + bundle check
[2020-12-02T09:41:42.361Z] Warning: the running version of Bundler (2.0.2) is older than the version that created the lockfile (2.1.4). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.
[2020-12-02T09:41:42.622Z] The following gems are missing
[2020-12-02T09:41:42.622Z]  * rake (13.0.1)
[2020-12-02T09:41:42.622Z]  * builder (3.2.4)
[2020-12-02T09:41:42.622Z]  * diff-lcs (1.4.4)
[2020-12-02T09:41:42.622Z]  * ruby2_keywords (0.0.2)
[2020-12-02T09:41:42.622Z]  * mustermann (1.1.1)
[2020-12-02T09:41:42.622Z]  * rack (2.2.3)
[2020-12-02T09:41:42.622Z]  * rack-protection (2.1.0)
[2020-12-02T09:41:42.622Z]  * rack-test (1.1.0)
[2020-12-02T09:41:42.622Z]  * rspec-support (3.10.0)
[2020-12-02T09:41:42.622Z]  * rspec-core (3.10.0)
[2020-12-02T09:41:42.622Z]  * rspec-expectations (3.10.0)
[2020-12-02T09:41:42.622Z]  * rspec-mocks (3.10.0)
[2020-12-02T09:41:42.622Z]  * rspec (3.10.0)
[2020-12-02T09:41:42.622Z]  * tilt (2.0.10)
[2020-12-02T09:41:42.622Z]  * sinatra (2.1.0)
[2020-12-02T09:41:42.622Z]  * yarjuf (2.0.0)
[2020-12-02T09:41:42.622Z] Install missing gems with `bundle install`
[2020-12-02T09:41:42.622Z] + rm Gemfile.lock
[2020-12-02T09:41:42.622Z] + bundle
[2020-12-02T09:41:44.537Z] Fetching gem metadata from https://rubygems.org/...
[2020-12-02T09:41:44.537Z] Resolving dependencies...
[2020-12-02T09:41:44.537Z] Fetching rake 13.0.1
[2020-12-02T09:41:44.537Z] Installing rake 13.0.1
[2020-12-02T09:41:44.802Z] Fetching builder 3.2.4
[2020-12-02T09:41:44.802Z] Using bundler 2.0.2
[2020-12-02T09:41:44.802Z] Fetching diff-lcs 1.4.4
[2020-12-02T09:41:44.802Z] Using ecs-logging 0.1.0 from source at `.`
[2020-12-02T09:41:44.802Z] Fetching ruby2_keywords 0.0.2
[2020-12-02T09:41:44.802Z] Installing ruby2_keywords 0.0.2
[2020-12-02T09:41:44.802Z] Installing builder 3.2.4
[2020-12-02T09:41:44.802Z] Fetching rack 2.2.3
[2020-12-02T09:41:44.802Z] Fetching rspec-support 3.10.0
[2020-12-02T09:41:44.802Z] Installing diff-lcs 1.4.4
[2020-12-02T09:41:45.072Z] Installing rspec-support 3.10.0
[2020-12-02T09:41:45.072Z] Fetching tilt 2.0.10
[2020-12-02T09:41:45.072Z] Installing rack 2.2.3
[2020-12-02T09:41:45.072Z] Fetching mustermann 1.1.1
[2020-12-02T09:41:45.072Z] Installing tilt 2.0.10
[2020-12-02T09:41:45.339Z] Installing mustermann 1.1.1
[2020-12-02T09:41:45.339Z] Fetching rspec-core 3.10.0
[2020-12-02T09:41:45.339Z] Fetching rspec-expectations 3.10.0
[2020-12-02T09:41:45.339Z] Fetching rspec-mocks 3.10.0
[2020-12-02T09:41:45.339Z] Installing rspec-core 3.10.0
[2020-12-02T09:41:45.339Z] Installing rspec-expectations 3.10.0
[2020-12-02T09:41:45.339Z] Installing rspec-mocks 3.10.0
[2020-12-02T09:41:45.604Z] Fetching rack-protection 2.1.0
[2020-12-02T09:41:45.604Z] Installing rack-protection 2.1.0
[2020-12-02T09:41:45.604Z] Fetching rack-test 1.1.0
[2020-12-02T09:41:45.604Z] Fetching rspec 3.10.0
[2020-12-02T09:41:45.604Z] Fetching sinatra 2.1.0
[2020-12-02T09:41:45.604Z] Installing rack-test 1.1.0
[2020-12-02T09:41:45.604Z] Installing rspec 3.10.0
[2020-12-02T09:41:45.604Z] Fetching yarjuf 2.0.0
[2020-12-02T09:41:45.868Z] Installing yarjuf 2.0.0
[2020-12-02T09:41:45.868Z] Installing sinatra 2.1.0
[2020-12-02T09:41:45.868Z] Bundle complete! 6 Gemfile dependencies, 18 gems now installed.
[2020-12-02T09:41:45.868Z] Bundled gems are installed into `/vendor/ruby_2.4`
[2020-12-02T09:41:45.868Z] + [[ '' == spec/* ]]
[2020-12-02T09:41:45.868Z] + [[ 0 == 0 ]]
[2020-12-02T09:41:45.868Z] + runRspec
[2020-12-02T09:41:45.868Z] + local case=
[2020-12-02T09:41:45.868Z] + local bn=
[2020-12-02T09:41:45.868Z] + '[' -n '' ']'
[2020-12-02T09:41:45.868Z] + '[' -n 2.4.10 ']'
[2020-12-02T09:41:45.868Z] + bn=2.4.10-
[2020-12-02T09:41:45.868Z] + '[' -n sinatra-2.0 ']'
[2020-12-02T09:41:45.868Z] + bn=sinatra-2.0-2.4.10-
[2020-12-02T09:41:45.868Z] + bundle exec rspec -f progress -r yarjuf -f JUnit -o spec/junit-reports/sinatra-2.0-2.4.10-ruby-agent-junit.xml
[2020-12-02T09:41:46.816Z] .....
[2020-12-02T09:41:46.816Z] 
[2020-12-02T09:41:46.816Z] Finished in 0.0194 seconds (files took 0.36597 seconds to load)
[2020-12-02T09:41:46.816Z] 5 examples, 0 failures
[2020-12-02T09:41:46.816Z] 
[2020-12-02T09:41:46.816Z] + exit 0
[2020-12-02T09:41:48.018Z] Post stage
[2020-12-02T09:41:48.034Z] Recording test results
[2020-12-02T09:41:48.786Z] Running on Jenkins in /var/lib/jenkins/workspace/t-ruby_ecs-logging-ruby-mbp_PR-4
[2020-12-02T09:41:48.819Z] [INFO] getVaultSecret: Getting secrets
[2020-12-02T09:41:49.001Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-12-02T09:41:49.592Z] + chmod 755 generate-build-data.sh
[2020-12-02T09:41:49.592Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/ecs-logging-ruby-mbp/PR-4/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/ecs-logging-ruby-mbp/PR-4/runs/18 FAILURE 659131
[2020-12-02T09:41:50.143Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/ecs-logging-ruby-mbp/PR-4/runs/18/steps/?limit=10000 -o steps-info.json

@v1v
Copy link
Member Author

v1v commented Nov 30, 2020

If rspec is not installed then bin/* does not report an error:

image

Comment on lines +48 to +49
name 'RUBY_VERSION'
values 'ruby:2.7', 'ruby:2.6', 'ruby:2.5', 'ruby:2.4', 'ruby:2.3', 'jruby:9.2', 'jruby:9.1'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List of supported ruby versions to be tested:

.ci/Jenkinsfile Outdated
Comment on lines 52 to 53
name 'FRAMEWORK'
values 'rails-6.0', 'rails-5.2', 'rails-5.1', 'rails-4.2', 'sinatra-2.0', 'sinatra-1.4'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List of supported frameworks to be tested:

ENV PATH=/app/bin:$BUNDLE_BIN:$PATH

ENV FRAMEWORKS $FRAMEWORKS
ENV RUBY_IMAGE $RUBY_IMAGE
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will help to generate junit files with the ruby_image value in their names

bin/dev Outdated
' docker-compose run' \
' --rm' \
" specs #{ARGV.join}"
exit $?.exitstatus unless $?.success?
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If something goes wrong then this will help to report back the error

#!/bin/bash
set -x

runRspec(){
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delegate the how to the function then junit reporting are generated with a different name

Comment on lines +27 to +29
# Copy cached folder to speed up docker containers
COPY vendor /vendor
RUN chown -R $USER_ID_GROUP /vendor
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required to use the cached artifacts if required, otherwise the docker build uses the VENDOR_PATH but the docker run overrides it, as a consequence it fails in the CI.

When building a docker image there is now way to mount volumes but using the COPY, this will help to reuse the existing local cache artifacts.

@v1v v1v marked this pull request as ready for review December 1, 2020 13:37
@v1v v1v requested review from a team December 1, 2020 13:38
Copy link
Contributor

@mikker mikker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is SO GOOD, Victor! Can we bring some of this back to the agent's setup? The split GH checks, the caching changes, whatever else you have in store 😊

Co-authored-by: Mikkel Malmberg <mikkel@brnbw.com>
@mikker
Copy link
Contributor

mikker commented Dec 2, 2020

@v1v
Copy link
Member Author

v1v commented Dec 2, 2020

@mikker , see the comment -> #4 (comment)

It contains all the digested details from the build, so it might be easier

@mikker mikker merged commit 721ec47 into elastic:master Dec 2, 2020
@v1v v1v deleted the feature/jenkinsfile branch December 2, 2020 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants